I joined, after seeing the latest video on Star Rod, wow, and am a very big Paper Mario 64 fan!   Anyways, I noticed in the post:
"Music/sound editing. If someone cracks the format, I will add it."  

Punk7890 and I have *mostly* cracked the format on both of them.  The midis there are some commands that are unknown though, but most of it is identified.  You can check the box for debug parse, on ripping, to get a file showing the format (or just look at the source code).  Also, the exact tune on instruments is not quite cooperating in the soundfont when playing back midis.  

Music Rip - music is semi-complicated, also not sure why each song has multiple songs, and sub-songs concatenated:
http://goldeneyevault.com/viewfile.php?id=211
(and some notes:  https://drive.google.com/file/d/0Bz8PWXTWctXKVFlCUi1tVFRlaGM/view?usp=sharing)

Paper Mario
801DA070 Music
81159B03 XXXX Music Mod
801E8A40 Table of music
801D588C Table of instrument
801D57A0 is start of data looks like
(0th value & 0x70) >> 2
30 goes to C, loaded off of table at 801D6A8C + this shifted value, start points

Banks # starts
0: Empty
1: MAR0CR
2: Empty
3: GM01CR
4: BT20CR
5: ENE0CR
6: BT2CC
7: Empty

Sound Rip:
http://goldeneyevault.com/viewfile.php?id=212

Soundfont Rips (per midi) - click Write Combined Soundbanks after loading, tuning still not quite right overall:
http://goldeneyevault.com/viewfile.php?id=302

(June 18th, 2016, 06:52 am)SubDrag Wrote: I joined, after seeing the latest video on Star Rod, wow, and am a very big Paper Mario 64 fan!   Anyways, I noticed in the post:
"Music/sound editing. If someone cracks the format, I will add it."  

Punk7890 and I have *mostly* cracked the format on both of them.  The midis there are some commands that are unknown though, but most of it is identified.  You can check the box for debug parse, on ripping, to get a file showing the format (or just look at the source code).  Also, the exact tune on instruments is not quite cooperating in the soundfont when playing back midis.  

Music Rip - music is semi-complicated, also not sure why each song has multiple songs, and sub-songs concatenated:
http://goldeneyevault.com/viewfile.php?id=211
(and some notes: https://drive.google.com/file/d/0Bz8PWXTWctXKQmlhdDB5NmFuYXM/view )

Paper Mario
801DA070 Music
81159B03 XXXX Music Mod
801E8A40 Table of music
801D588C Table of instrument
801D57A0 is start of data looks like
(0th value & 0x70) >> 2
30 goes to C, loaded off of table at 801D6A8C + this shifted value, start points

Banks # starts
0: Empty
1: MAR0CR
2: Empty
3: GM01CR
4: BT20CR
5: ENE0CR
6: BT2CC
7: Empty

Sound Rip:
http://goldeneyevault.com/viewfile.php?id=212

Soundfont Rips (per midi) - click Write Combined Soundbanks after loading, tuning still not quite right overall:
http://goldeneyevault.com/viewfile.php?id=302

​
How come the weird noise thing... I was looking trough beta sounds and the only recognisable one was this: https://www.youtube.com/watch?v=gqcuyxoyElk Any way to fix the instruments weirdly pitched (One of the sounds sounded YV style so it might have been a beta of that)

We haven't been able to properly fix the tuning unfortunately, it's in a couple areas and haven't been able to figure out how to apply it properly.

Thanks, I hadn't seen this tool before. Now that I can see which BGM files correspond to which song, I have figured out how the music is arranged in the engine (ie, track ID 17 = Dry Dry Desert) and how the file table is organized. There is no reason we can't add custom music now. Wherever possible, I try to avoid looking at the data as it exists in memory since we are after the representation on the ROM. I have some notes I can share, but first I have a few questions. Disclaimer: I know very little about MIDI file formats.

First, evidently the BGM files store track information (like sheet music), and the BK files have samples or synthesizer settings of some sort. What do the MSEQ files do? Other file types are also PER (percussion), SEF (sound effect), and PRG (?).

Second, has anyone written tools that take MIDI files and convert them into BGM files? I think we can just use the existing BK files for now and support custom arrangements, but is there anything special we need to do to ensure compatibility? I'd really like to see a detailed breakdown of the BGM and BK formats.

I'm sure that music lookup table is in ROM as well, just was easier to see in RAM.

The only files I figured out were PER files (percussion lookup when notes are percussion on a drum track). I didn't make any tools to import to this BGM format because it is not 100% understood, and was focusing on ripping. Some commands are still unknown. I recommend taking a look at source code of all 3 tools relating to Paper Mario, and the "debug parse" the midi tool can export should be very helpful. There is some cool stuff like Toad Town has a song that has randomness in which data it uses. Also, not all the commands are known and flags on each track. Each song can use the instrument bank in header (custom tuning/volume, etc), drum bank in header (custom tuning/volume, etc), any instrument in sound bank, or any instrument in PER drum bank. The drum tracks are the only tracks that can use drums (flag on track), and the instruments on regular tracks. It's a complicated format, you'll have to really delve into what Punk7890 and I did. Especially the BGM files you should look at the debug parses the midi tool puts out if you check the box.

I'm not sure what the BK files are? I think the MSEQ files are effects, but not really delved into them.

(June 18th, 2016, 04:19 pm)clover Wrote: Thanks, I hadn't seen this tool before. Now that I can see which BGM files correspond to which song, I have figured out how the music is arranged in the engine (ie, track ID 17 = Dry Dry Desert) and how the file table is organized. There is no reason we can't add custom music now. Wherever possible, I try to avoid looking at the data as it exists in memory since we are after the representation on the ROM. I have some notes I can share, but first I have a few questions. Disclaimer: I know very little about MIDI file formats.

First, evidently the BGM files store track information (like sheet music), and the BK files have samples or synthesizer settings of some sort. What do the MSEQ files do? Other file types are also PER (percussion), SEF (sound effect), and PRG (?).

Second, has anyone written tools that take MIDI files and convert them into BGM files? I think we can just use the existing BK files for now and support custom arrangements, but is there anything special we need to do to ensure compatibility? I'd really like to see a detailed breakdown of the BGM and BK formats.

​

The BGM files you mentioned are the custom midi-like data that N64 Midi Tool converts to midi. It would be very tricky to convert midi to Paper Mario BGM files as there is a bunch of things to consider. I only worked on converting BGM to midi with SubDrag, but, in theory if converting midi to BGM was supported, you would need very small midis in order to fit in a sequence spot. BGM files are divided into Segments with a corresponding Sub-segment. Each Segment is generally a variation of the same song (take Normal Battle for instance, it has 3 segments. The first being the normal version of it, then the second segment is with a different intro). You would need to replace the Sub-segments which are very small most of the time (generally 8 or 16 bars). The second thing would be all of the proper commands, which shouldn't be much of a problem (tempo, volume, pan, reverb to name a few easy ones in theory). The more custom commands would be the hard part. One such command would be FEXXXXYY which is a subroutine. This command can save quite a bit of space. I'm not fully sure how it works, SubDrag should know more about that one. Another thing is volume in this game is handled really odd. There is a total of around 3 volume commands that get used frequently (volume in the header if loaded, volume in commands (E9 and EC are common). There is a total of 6 volume commands/header volumes. Third would be the instrument look up headers in each BGM file. Not every song has this and probably isn't all that necessary for importing to be honest. Fourth would be the track flags which I'm not fully sure what they all mean. A track flag with XX80 is a drum track. A drum track is filled with all drums over the entire keyboard. I've seen a few drums notes that were loaded in the header. What that means is somehow it combines new tuning info for that drum note / spot in the drum bank. I'm not sure how tuning works in Paper Mario yet. That's all I can think of at the moment. Proper timing may be a slight issue as there is certain delay slots that can extend really far. To get a better idea how this format works and better conceptual understanding, take a look at what N64 Midi Tool exports with the Debug Text File option.

The BK files you mentioned contain the sample information, loop points and point to its sample data. The tuning info / ADSR is stored else where.

MSEQ's are a sequence format for SFX. I didn't even bother with this. It plays bird chirps and such.

Here's my latest notes. I think Sub had a old one: https://drive.google.com/file/d/0Bz8PWXTWctXKVFlCUi1tVFRlaGM/view?usp=sharing

(June 19th, 2016, 05:10 am)punk7890 Wrote:

(June 18th, 2016, 04:19 pm)clover Wrote: ​
Thanks, I hadn't seen this tool before. Now that I can see which BGM files correspond to which song, I have figured out how the music is arranged in the engine (ie, track ID 17 = Dry Dry Desert) and how the file table is organized. There is no reason we can't add custom music now. Wherever possible, I try to avoid looking at the data as it exists in memory since we are after the representation on the ROM. I have some notes I can share, but first I have a few questions. Disclaimer: I know very little about MIDI file formats.

First, evidently the BGM files store track information (like sheet music), and the BK files have samples or synthesizer settings of some sort. What do the MSEQ files do? Other file types are also PER (percussion), SEF (sound effect), and PRG (?).

Second, has anyone written tools that take MIDI files and convert them into BGM files? I think we can just use the existing BK files for now and support custom arrangements, but is there anything special we need to do to ensure compatibility? I'd really like to see a detailed breakdown of the BGM and BK formats.

​

The BGM files you mentioned are the custom midi-like data that N64 Midi Tool converts to midi. It would be very tricky to convert midi to Paper Mario BGM files as there is a bunch of things to consider. I only worked on converting BGM to midi with SubDrag, but, in theory if converting midi to BGM was supported, you would need very small midis in order to fit in a sequence spot. BGM files are divided into Segments with a corresponding Sub-segment. Each Segment is generally a variation of the same song (take Normal Battle for instance, it has 3 segments. The first being the normal version of it, then the second segment is with a different intro). You would need to replace the Sub-segments which are very small most of the time (generally 8 or 16 bars). The second thing would be all of the proper commands, which shouldn't be much of a problem (tempo, volume, pan, reverb to name a few easy ones in theory). The more custom commands would be the hard part. One such command would be FEXXXXYY which is a subroutine. This command can save quite a bit of space. I'm not fully sure how it works, SubDrag should know more about that one. Another thing is volume in this game is handled really odd. There is a total of around 3 volume commands that get used frequently (volume in the header if loaded, volume in commands (E9 and EC are common). There is a total of 6 volume commands/header volumes. Third would be the instrument look up headers in each BGM file. Not every song has this and probably isn't all that necessary for importing to be honest. Fourth would be the track flags which I'm not fully sure what they all mean. A track flag with XX80 is a drum track. A drum track is filled with all drums over the entire keyboard. I've seen a few drums notes that were loaded in the header. What that means is somehow it combines new tuning info for that drum note / spot in the drum bank. I'm not sure how tuning works in Paper Mario yet. That's all I can think of at the moment. Proper timing may be a slight issue as there is certain delay slots that can extend really far. To get a better idea how this format works and better conceptual understanding, take a look at what N64 Midi Tool exports with the Debug Text File option.

The BK files you mentioned contain the sample information, loop points and point to its sample data. The tuning info / ADSR is stored else where.

MSEQ's are a sequence format for SFX. I didn't even bother with this. It plays bird chirps and such.

Here's my latest notes. I think Sub had a old one: https://drive.google.com/file/d/0Bz8PWXTWctXKVFlCUi1tVFRlaGM/view?usp=sharing

​

Is there some way to let the tool export directly raw BGM files (or hex)?

In N64 Midi Tool, click Export Bin. That will export the raw sequence from any game.

(September 1st, 2016, 09:29 pm)punk7890 Wrote: In N64 Midi Tool, click Export Bin. That will export the raw sequence from any game.

​

K ill check out later